Item pool design for computerized adaptive tests

Although item pools are critical to the proper functioning of computerized adaptive tests (CATs), there is very little in the research literature that indicates the desired features of an item pool. Most research articles use an existing item pool that was developed for other purposes. For example, Pastor, Dodd and Chang (2002) used items from NAEP; Wang and Kolen (2001) used an item pool from paper-and-pencil ACT Mathematics test forms. The standard texts on adaptive testing give little guidance. Flaugher (2000) indicates that “the item pool from which items are selected must contain high-quality items for many different levels of proficiency” (p. 38). Later in the chapter he provides an example with 200 items in the pool. Veldkamp and van der Linden (2000) give detailed guidance about designing an item pool once specifications have been developed, but they do not tell how to produce the specifications in the first place. Stocking (1994) suggested that item pools should contain six or seven times as many items as the length of the adaptive test, but this figure was based mostly on concerns over test security rather than the number and distribution of items needed to measure effectively. The information function for the item pool was of concern, but the criterion for information was based on the characteristics of previous paper-and-pencil forms. No specific procedures have been identified for developing the specifications for a computerized adaptive testing item pool.